1. Field of the Invention
The invention generally relates to apparatus and concomitant methods of providing security for computer systems, and particularly, to apparatus and concomitant methods for establishing secure access to computer application programs using a pseudo-terminal link.
2. Description of the Prior Art
"UNIX" (a registered trademark of UNIX System Laboratories, Inc.), operating systems generally refers to a family of computer operating systems. UNIX operating systems provide task dispensing, a file system, and input/output (I/O) control for computer systems. However, certain computer operating systems exist which utilize command structures that are similar to UNIX command structures but are not known as UNIX operating systems per se, i.e., those operating systems being known as "UNIX-like" operating systems. With this in mind, the following discussion of UNIX-based computer systems, i.e., computer systems using a UNIX operating system, encompasses computer systems which utilize UNIX-like operating systems as well as UNIX operating systems.
Generally, a UNIX system has its software logically arranged in a hierarchical manner. Within this hierarchy, a lowest level contains device input/output (I/O) programs known as device drivers which interact directly with hardware such as printers, terminals, memory, and the like; a higher level contains a file system for storing and manipulating files; and the highest level, known as the application level, contains application programs which interact with the hardware through the operating system. The entire operating system, containing the file system and device drivers, is commonly known as a "kernel". In addition to the file system and device drivers, the kernel also provides ancillary services such as memory management and central processing unit (CPU) scheduling.
The application level typically contains executable files or application programs which request services from the kernel using instructions known as "system calls". Using system calls, an application program can open assorted files to retrieve information, start other executable routines, produce data and retrieve data.
In a UNIX environment, file security is generally provided by the kernel through a technique, described below, that alters a set of file protection flags or codes. These flags control who may access particular files. However, some types of files, in particular pseudo-teletype (pseudo-tty) character-special files, must be accessible, at least initially, to any process (an executed application program) executed on the UNIX-based computer. Generally, pseudo-tty character-special files facilitate network communications between computers. Consequently, to enable any process to communicate with other computers, the file protection flag settings associated with pseudo-tty character-special files generally are set to allow any process to initially access the files. Since the kernel permits any process to access these files, once the file is opened, an additional level of security must be provided by each process that accesses these files. Typically, this security is effectuated by a subroutine within the application program that instructs the kernel to alter the protection flags of each open pseudo-tty character-special file.
The application programs used to facilitate network communications are known as "communications application" programs. To achieve the additional level of security, each communications application program must be executed as an authorized (typically referred to as a "SUPERUSER") process. As such, to provide security, the authorized process can subsequently request the kernel to alter the file protection flags and ownership of the pseudo-tty character-special files.
In some instances, these authorized processes do not provide adequate security, allowing malicious users to receive data intended to flow into, or out from, the UNIX-based computer through its terminal connection to the network. Specifically, when a communications application program, such as telnet, rlogin, or X-windows, uses a specific type of character-special file known as a pseudo-terminal to facilitate connections between a terminal and an application program, certain critical security issues arise. Before discussing the particulars of these security issues, the complexity of these issues necessitates a discussion of the basics of pseudo-terminal utilization.
A "pseudo-terminal" is a special type of pseudo-tty character-special file controlled by a set of device driver routines (hereinafter the combination of the device driver routines with the necessary pseudo-tty character-special file is referred to as a pseudo-terminal (pty) device driver). The pty device driver permits a communications application program (hereinafter referred to as a "communications application") to interface to other application programs as if that communications application was an actual terminal connected to the UNIX-based computer. A specific example of a communications application is "telnet". Specifically, if a computer connected to a network at a remote site, known as a remote computer, desires to connect to a UNIX-based host computer that is connected to the same network, the remote computer connects to the host computer via telnet. In this regard, using conventional communications software, the remote computer contacts the telnet communications application within the host computer and requests access thereto. Two pty device drivers within the host computer are used to facilitate a pseudo-terminal connection between the telnet communications application being accessed by the remote computer and any user application within the host computer. Subsequently, the host computer, and any user application executed thereupon, views the remote computer as a terminal that is directly connected to the host computer. The remote computer and the intervening network become transparent to the user application.
More specifically, the process by which a remote login from a remote computer to a UNIX-based Host computer is accomplished begins with the remote computer accessing, via the network, a remote login communications application resident on the host computer. Typically, the remote login application is either "riogin", which assumes the host and remote computers are both UNIX-based computers or "telnet" which is a standard INTERNET application that most Transmission Control Protocol/Internet Protocol (TCP/IP) implementations support regardless of the operating system of the host computer.
Once the remote computer has accessed a remote login communications application, the application executes a "fork" system call. The "fork" system call creates a copy of the process that was executing, in this instance, a copy of the remote login application. Consequently, the copy of the remote login application, known as a child process, continues to interact, via the network, with the remote computer. Meanwhile, the remote login application, known as a parent process, becomes dormant and awaits the next remote computer request for access to the host computer.
The child process (hereinafter referred to as a "server process") opens a pair of pseudo-tty character-special files known as pseudo-terminal (pty) files. The pair of pty files comprise a pty "master" file and a pty "slave" file. The pty master file is logically associated with the pty slave file; a so-called pseudo-terminal link connects the two files together. In this manner, commands and other data entering the master file pass to the slave file and vice versa. When the pty files are opened, each is controlled by a set of device driver routines such that the combination of each of the pty files with their respective device driver routines forms a pty logical link between the pty files.
Next, the server process executes a "fork" system call which creates a copy (a child) of the server process. Now, each of these processes, the server process and its child, has an open connection to both the pty master and slave files. To end the dual connection, the server process closes its connection to the pty slave file. Similarly, the child of the server process closes its connection to the pty master file. As a result, a logical communications path exists between the server process and its child. The child process views this path as a terminal connection, i.e., a connection from the child process directly to a terminal. The child process then issues an "exec" system call for a login shell application to facilitate remote computer login to the host computer. In this manner, the pty slave file becomes connected to the login shell application. Consequently, the remote computer will operate in a similar manner to a terminal connected to the host computer. Any information typed on the remote computer keyboard forms an input to the login shell application and any information returned from that application is transferred to a display on the remote computer. The user of the remote computer may now proceed to login to the host computer and access information therein.
To facilitate security of the communication between the remote computer and the host computer, a communication application such as telnet performs certain security precautions after opening the pty files. Specifically, after the communication application in the form of the server process opens the pty master and slave files, the server process alters access permission code and ownership of the pty slave file to restrict subsequent access to this file by other users. To enable the access permission code and ownership to be changed, each application which alters the pty slave file access and ownership must have superuser status.
When the server process first opens a pty file, the permission codes, are typically "rw.rw.rw", wherein each field from left to right corresponds to a permission code assigned to the current owner, the group in which the owner is a member, and the rest of the world. Within the permission codes, "r" indicates that a user can read the file; "w" indicates that a user can write to the file; "x" indicates that the file can be executed. Note that in this particular example "x" is not used, i.e., the file is not executable, therefore, the "x" is replaced with a dot (.). Typically, the initial owner of the pty files is the superuser name, "root", and the permission codes enable any user to access the files on demand. Therefore, in the case of pty files, both the pty master file and the pty slave file can initially be accessed by any process.
To establish a secure pty link, once opened, the server process changes: (a) the access permission codes of the pty slave file to "rw..w. . . .", and (b) the owner of the pty slave file to an effective user identification (UID) of the server process. After alteration of the codes and owner, only the present user can read and write to the opened pty slave file. Additionally, those users in the group which the implementation has defined are permitted to write to the file. However, those in the outside world do not have access to the open pty slave file at all. This access restriction presents a limited amount of security. However, since the application program is responsible for securing the open pty slave file, it is foreseeable that some applications may not include such security measures. Thus, the permission codes and ownership would not be altered and consequently, open pty files would be accessible by all users at all times.
To circumvent the limited security afforded by altering the access permission codes and owner of the pty slave file, an unauthorized user can exploit so-called "timing windows" that exist between opening the pty files and changing the permission codes and ownership. A timing window is a time interval that occurs between the act of opening the files and the act of changing the permission codes and the ownership of the open files. Typically, the acts of opening a file and changing the ownership and permission codes are non-atomic operations. Generally speaking, a non-atomic operation is an operation which accomplishes a particular task using multiple system calls. The existence of timing windows has traditionally been referred to as the "time of access versus time of check" problem. Because of the existence of timing windows, an unauthorized user can invoke an unauthorized process to facilitate compromising file security by opening a pty file prior to the ownership/permission code change. Once a file has been opened by a process, a subsequent change in permissions or ownership of the opened file does not affect access by the process that previously opened the file. Consequently, once the unauthorized process has accessed the pty slave file, the authorized process cannot disconnect the unauthorized process from the file by changing the permission codes and ownership of that file. Thus, the unauthorized user remains connected to the pty slave file. In this manner, the unauthorized process compromises the security of the link between the pty master file and the pty slave file. Therefore, the unauthorized process may receive any data transmitted by the remote computer, including user id and password information. This information can be stored by the unauthorized user for later use to compromise security of the host computer. Alternatively, the unauthorized process can send misleading data to the remote computer. In this manner, the unauthorized process can request the user of the remote computer to transmit his or her user id, password or other confidential information to the unauthorized process. Such malicious action would thus compromise security of the host computer.
Thus, a need exists in the art for a method and apparatus for creating a secure link between a pty master file and a pty slave file. Additionally, such a method and apparatus should relieve application programs of the responsibility of security and remove the necessity of the application running as a superuser, this method and apparatus should reside within the kernel.